An example usage of strymread

Analyzing CAN data logged from Giraffee and Panda

Prerequisite

Install strym. See documentation for installation instruction: https://jmscslgroup.github.io/strym/installation.html

We will read the CAN data file which is recorded via libpanda using comma.ai Panda devices. You must supply a DBC file for decoding hex messes. If you don’t provide DBC file, strymread defaults to using RAV4 2019 DBC file which is pre-packaged with strym. In this example, we won’t be supplying DBC file as data that we are going to read is from Toyota RAV4. strymread will use default DBC file. Further, naming of CAN CSV file must have VIN number and strymread will guess whihc DBC file to use.

[1]:
import strym
from strym import strymread
import matplotlib.pyplot as plt
import pandas as pd
import numpy as np
csvfile = '../PandaData/2020_06_01/2020-06-01-13-01-36_2T3Y1RFV8KC014025_CAN_Messages.csv'
r =strymread(csvfile=csvfile)
Loading BokehJS ...

We will visualize the counts of all messages

The plot is split into several subplots for brevity. The function returns a count dataframe with Message count by BUS IDs and total counts

[2]:
count = r.count(plot=True)
/home/ivory/anaconda3/envs/dbn/lib/python3.7/site-packages/strym-0.3.5-py3.7.egg/strym/strymread.py:553: UserWarning:

Matplotlib is currently using module://ipykernel.pylab.backend_inline, which is a non-GUI backend, so cannot show the figure.

_images/Strymread_example_4_1.png
[3]:
count
[3]:
MessageID Counts_Bus_0 Counts_Bus_1 Counts_Bus_2 TotalCount
36 36 54926 0 54926 109852
37 37 109852 0 109852 219704
170 170 109852 0 109852 219704
180 180 54926 0 54926 109852
186 186 36617 0 36617 73234
... ... ... ... ... ...
1990 1990 45 0 45 90
1994 1994 27 0 27 54
1998 1998 45 0 45 90
2004 2004 52 0 52 104
2012 2012 52 0 52 104

209 rows × 5 columns

Let’s plot a few important data

Speed

We first make static plot by setting strym.config['interactive']=False

[4]:
strym.config['interactive']=False
speed = r.speed()
strymread.plt_ts(speed, title="Speed Plot")
/home/ivory/anaconda3/envs/dbn/lib/python3.7/site-packages/strym-0.3.5-py3.7.egg/strym/strymread.py:2855: UserWarning:

Matplotlib is currently using module://ipykernel.pylab.backend_inline, which is a non-GUI backend, so cannot show the figure.

_images/Strymread_example_8_1.png

Now, we create interactive plot

[6]:
strym.config['interactive']=True
speed = r.speed()
strymread.plt_ts(speed, title="Speed Plot")

Lets visualize a message with given message ID and signal ID

[7]:
msg869 = r.get_ts(869, 6)
msg869
[7]:
Time Message Bus
Clock
2020-06-01 20:01:37.706951857 1.591042e+09 252 2
2020-06-01 20:01:37.706951857 1.591042e+09 252 0
2020-06-01 20:01:37.727082968 1.591042e+09 252 2
2020-06-01 20:01:37.727082968 1.591042e+09 252 0
2020-06-01 20:01:37.747102022 1.591042e+09 252 2
... ... ... ...
2020-06-01 20:19:54.863569021 1.591043e+09 2 0
2020-06-01 20:19:55.064127922 1.591043e+09 2 2
2020-06-01 20:19:55.064127922 1.591043e+09 2 0
2020-06-01 20:19:55.264127970 1.591043e+09 2 2
2020-06-01 20:19:55.264127970 1.591043e+09 2 0

10984 rows × 3 columns

[8]:
strymread.plt_ts(msg869, title="Message 869, Signal 6")

Rate Statistics for every message ID

[9]:
u = r.frequency()
u
[9]:
MessageID MeanRate MedianRate RateStd MaxRate MinRate RateIQR
0 36 457.418698 50.127928 1461.255929 12985.461300 11.852837 1.420878
1 37 518.697935 100.563537 1445.840868 11618.570637 13.396566 5.826275
2 170 390.135893 100.301409 1224.572556 11522.813187 13.373116 4.687697
3 180 546.361381 50.153103 1565.040143 12192.744186 12.956059 1.372874
4 186 418.632506 33.459407 1432.105869 11491.243836 11.474896 1.259073
... ... ... ... ... ... ... ...
204 1990 15.867022 10.013666 16.021242 72.290658 0.004203 7.691075
205 1994 785.949826 67.864416 1885.388254 6574.144201 0.033738 102.817044
206 1998 155.187068 10.118899 919.100583 6250.825633 0.004203 14.288731
207 2004 434.944156 10.023070 1506.322766 6061.132948 0.017528 18.321202
208 2012 514.649557 10.551886 1621.569966 6403.517557 0.017533 13.465362

209 rows × 7 columns

Synchronize Two Time Series messages and resample with a fixed datarate

It means first time of speed is earlier than yaw in time-series data so we have to interpolate speed value at yaw’s first time. We will use linear interpolation.

Linear interpolation formula is

\[X_i = \cfrac{X_A - X_B}{a-b}(i-b) + X_B\]
[10]:
ts_yaw_rate = r.yaw_rate()
ts_speed = r.speed()

# integrate yaw rate to get the heading
ts_yaw = strymread.integrate(ts_yaw_rate)
[11]:
strymread.plt_ts(ts_yaw, title="Yaw")

Plot the trajectory of vehice based on kinematic model using yaw rate and speed

[15]:
T = r.trajectory()
[16]:
fig, ax = strymread.create_fig(1)
ax[0].scatter(x = 'X', y = 'Y', c = 'Time', data = T, s = 1)
ax[0].set_title("Vehicle's Trajectory")
plt.show()
_images/Strymread_example_26_0.png
[17]:
T[1:7500]
[17]:
Time X Y Vx Vy
1 1.591042e+09 0.076889 0.000001 3.844444 0.000055
2 1.591042e+09 0.159306 0.000071 4.120851 0.003512
3 1.591042e+09 0.246237 0.000209 4.346545 0.006890
4 1.591042e+09 0.336663 0.000399 4.521301 0.009474
5 1.591042e+09 0.429092 0.000633 4.621473 0.011701
... ... ... ... ... ...
7495 1.591042e+09 1505.749024 99.366560 16.675783 -1.306036
7496 1.591042e+09 1506.082129 99.340442 16.655216 -1.305881
7497 1.591042e+09 1506.414734 99.314334 16.630274 -1.305411
7498 1.591042e+09 1506.746662 99.288252 16.596424 -1.304107
7499 1.591042e+09 1507.078212 99.262189 16.577464 -1.303143

7499 rows × 5 columns

Get the meta data about driving

driving_characteristics function will give dictionary formatted meta data of the drive being analyzed.

[18]:
metadata = r.driving_characteristics()
[19]:
metadata
[19]:
{'filename': '../PandaData/2020_06_01/2020-06-01-13-01-36_2T3Y1RFV8KC014025_CAN_Messages.csv',
 'dbcfile': '/home/ivory/anaconda3/envs/dbn/lib/python3.7/site-packages/strym-0.3.5-py3.7.egg/strym/dbc/toyota_rav4_2019.dbc',
 'distance_meters': 14631.19081597858,
 'distance_km': 14.631190815978579,
 'distance_miles': 9.09142307777013,
 'start_time': 'Mon Jun  1 13:01:37 2020',
 'end_time': 'Mon Jun  1 13:19:55 2020',
 'trip_time': 1097.749980211258}

Get the distribution of data

We will show the distribution of speed data

[20]:
speed = r.speed()
strymread.violinplot(speed['Message'], title="Speed Data")
/home/ivory/anaconda3/envs/dbn/lib/python3.7/site-packages/seaborn/_core.py:1296: UserWarning:

Horizontal orientation ignored with only `y` specified.

_images/Strymread_example_32_1.png

Rate Analysis for specific messages

Let’s say we want to analyze data throughput aka rate for Radar data. For this specific example, we will be looking at TRACK_A_0. For that we will call the appropriate function to retreive longitudinal data corresponding to TRACK_A_0.

[21]:
long_dist = r.long_dist(track_id=0) # I want to analyze rate for TRACK_A_0 only
long_dist
strymread.ranalyze(long_dist, title='Longitudinal Distance Data: TRACK A 0')
Analyzing Timestamp and Data Rate of Longitudinal Distance Data: TRACK A 0
Interquartile Range of Rate for Longitudinal Distance Data: TRACK A 0 is 0.18910648197294577
_images/Strymread_example_34_1.png

The above plot shows that, we receive RADAR data on TRACK A 0 at roughly 20 Hz in more or less consistent manner.

Extract Data for use in MATLAB

Often we want to extract data for analysis in MATLAB or let say your collaborator is not well-versed in Python but want data suitable for Analysis in MATLAB, in that case, we can extract data for use in MATLAB and it will save the data as .mat file.

[ ]:
files = r.export2mat(force_rewrite=True) #force_rewrite overwrites an already existing mat file if you had one previously

The mat file save has the same name as that of original csvfile passed to strymread object but with extension .mat